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METHOD, STORAGE MEDIUM, AND 
COMPOSITE REALITY EXHIBITION DEVICE 

(57) Abstract: 

PROBLEM TO BE SOLVED: To provide the device 
exhibiting a cooperative game by plural players in a sense 
of composite virtual reality. 

SOLUTION: The composite reality exhibiting device that 
generates a 3-dimension virtual image and displays the 
image on a see-through type display device so as to allow 
plural players to make a cooperative game in a composite 
reality environment is provided with a CCD camera 230 
that detects a position of a mallet of plural players and 
sensors 220 (220R, 220L) sensing visual positions in an 
environment of the cooperative game, generates a 
3-dimension virtual image denoting a result of the 
cooperative game sensed by sensors 220 and viewed 
from the visual point of each player in the cooperative 
game progressed in response to a change in the position 
of the mallet detected by the CCD camera 230 and 
displays the image on each see-through type display 
device. 
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X 


hy' 


= M T 


y 


_ h _ 







[0 0 4 4] MiteWkm&Xtr—^fr&n 

ZkL&Jr* V^ls—isa >-r^fc«6C0 3 X 3 CD^Wffn 
g£»T&£o Xf 7^S 2 2 0-C»6ftfcffi«fil 

( X * , y f ) ay-AttiMfsas o 3 o^asn 

tf, Xry^S2 1 6T r^p— /\*;U««T©flSSRj * 
fr^o ryp-;N*;u®«cTCO^j ni/yhtfS,^ 
o/c^^«, Xfy 2 2 0 TtO«attt6r-^ 



(7) 



4$B8¥ 1 1 -8 8 9 1 3 



11 



12 



[0 0 4 5] S90tt7l/^; hftD-*;HH«T«*-r 

(X-T-^y^S 2 1 2 0)l¥Jn) fiU Z<D 

£jg£n£*££ (2A+D x (2B+D mm<D& 

[0 0 4 6] 
[R3] 

x = [I\-A. I' x + A] 10 
y= [r y -A f I' y + B 3 

[0 0 4 7] uCT, I«, I,tt«SR««Ii4>a>ffijR 
(Dj&JifilT&O, A, B 

xf77 p s2 3 o 'Xy^'ytfs 2 2 ' o T^ea n&ie"" 

JglHtt^O^rTOBi* (x, y) K^^T, 4#®<£>I¥ffi 
fills (x, y) je©*fr*«JE^*fc<0*»HJT 

[0 0 4 8] 1P%, Xf7yS 2 3 2t?lt SflUff I $ 

*8<g£IBtgr-r&o ^iiftlilsKOxjagSlSJfctf 
yffi«tt*U^X^SUMxR«SUMyfc»3»8a«"TS. BP 

[0 0 4 9] 
[R4] 

N = N+ 1 
SUMx= SUMx+ x 
SUMy= SUMy+ y 

[0 0 5 0] Xfy^S 2 3 0*»7n^ 

8 1 2B««*mUy h*5(DM3t©/^->l: 

SUMx, SUMy**»Sft*. N=0T»ntfXf^^S2 3 40 
6TJfg^ "Not Found" ^BJASnS. N>0T»ft 
tf, 7 1/ y h & b ^ OM^^o fc©T» D , Xt7 
yS 2 3 8T, VU^hCOfi:® (I., Iy) &s 

[0 0 5 1] 

»5] 

SUMx 
x ~ N 

= SUMy 
y N 

[0 0 5 2] fcffi-3T«*T*. ^E-bT, Vl/7hfiS 50 



(I.. It) *T—^««*IC*ail/fc««ll** 
To 810HH 7fy^S2 16©yp-/^SS» 

[0 0 5 3] 
[R6] 

I (x, y) lx>0, x< Width, x = nC, 
y>0, y<Height, y=mD 
(fcCLn, mli&») I 

[0054] ^sis-r^ni^cofpT, &WL<Dmmm i s 

^>»*fflftUS?X^Maxt|2«"r*. C, Dtt 

«5R<0ffi;5*ft#££RT*O, Width* J;tfHeighttt 
^<2£St£8 1 SHfcSt. BP*. Xf7^S242 
«**Ii3&«H«[Eitl^^X^ICaxfc:iaitSnT^S 

fr^tzUZ& Xrv^S 2 4 4t, 

[0 0 5 5] 
[R7] 

Max= I s ( x f y) 
I 



I ,= y 

[0 0 5 6] <hT£o Xf7yS 2 4 6*eH j/D— A 
;i/«SBTat5*^fc«fe^Uy hSU^H* (I., I 
t ) ^I$Xf7^S 2 2 0[:It o ^Ocfc^tdb 

ffi«*fc«*bfcfc<0£y-Att»*3«5 0 3 OtCig 
"To 

<y-A«tt«a> ^1 3Htt, **i»»Ol7*y 

rau f-y^ i o o o (Z)±o 2 ^%¥®±^i^n, 

x, yfS^r^T^o &H<D 2 ^0)lSSW^-;i/7 

^*> 1 2 0 0 L, 1 2 0 0 R(h, 81 3eUC*5^T±T 
*lSll:Ktt6nfcfifiB!ll 300 a, 1 3 0 0 b <h£ 

tn. KS«i-;^>i 2 0 0 L, nooRi 

M69Hl 3 0 0 a, 1 3 0 0 btft ^©IfiflittK 

Tf, 7 1/7 h2 6 OR, 2 6 0 L <D#if|lCjS bt, My 
^7 1 5 0 0 (DiRlBffiVkifi&Wl-r&o 

[0 0 5 7] /Vy^l 5 0 0H 3Rft(fcS<Bffi«til IB P 
p£**11MBvp4:£*rU £7l/yh2 6 0 LBSfitt 
H<Z)Jffi«tiS«PsL t^S^fflv S L S71/ 7 h 

2 6 o Rttaatt«©«*flWBPsi ta«««vsi tt- 

JI14BH y-Att»»l»5 0 3 0ICitf 
[0 0 5 8] Xf7^S 1 0lC*5^T, /V;?1500 



(8) 

13 

xtexf>f v*nzmtzz>£5z3:&&ffi&*ffo* hi%, 
N&gff*«wgB5 o i o&mmi<tz&-?i/v h<D&m 

[0 0 5 9] Xf7yS12H y-A"C<D)Mfc#«fe£ 
T£ (Xr^^S 5 0T-^3MMt^) i£coBB 
14. A t PSWJttfc^fraft*. T5<h> Tst-v^S 1 2 

•ch, aw ^ 

[0 0 6 0] i0 
[R8] 

P,= PpO+V pfl ■ A t 

[0 0 6 1] {cMfr^tl^o «0)(lffill&CX«lfflSftK£ 
[0 0 6 2] 

_[»9J 

- P p = Pp+Vp - A t • - ■• - 

[0 0 6 3] ^$n^c xf^ys i 4 m H&r£ 

110 OLtT^TSffiBJCft^^g^SH^S. /W 

i7 1 5 o oa**^ 7^110 o Lt^FST^ttBt;:* 

*ttt, fi^lz-t 2 0 0 0*«Vl/7 h 2 6 0 LS/Vy 

1 5 0 0©iIft£RtESii-*fc«>K:* 
XfyyS18T, /VyiriS 00©a«v P OxJ|Sl 
a«E«»«>W#«K«ES-e:T, Xf^ys 2 Ofciitr. 
[0 0 6 4] *(CjSflEvp<Ox*|pjSKric»©?gF^- 56? 

[0 0 6 5^ 

"Trioi 

P P = - P PI +VSL« 

[0 0 6 6] <hlT, rty&lfl* Ts^J V 9 <£>}£f£ilffi 
a^tco/t^^firB^&yu—^^X^^ y 2 1 1 0 0 L 

tT^-rsttB^fe^a^ (xf^ys 1 6tno> \z 

tt, -t^SSX-x-y^S 2 0 (Cittfo XfyyS2 0t 40 
tt, /t *y ^ ©ffiB Pin #<K«13 1 3 0 0 aXttl30 
0 b tff^-rsffiHtCftS^S^SH^*. XtW^S 
2 O<O*M»rrt*YES<0«£fc:WU XfyyS2 2t/V; 

[0 0 6 7] ^KXt^^S 2 4T, 3Htt<B/V.y ^tt 

£>o YESO«^lCH Xf7^S2 6r«fiB©yi/ 
— V\ IP%, * (#2) -/U—VO#^^llP^T^o X 

3^±Tfcn«y-A«:*IT-r^o Xf 56? 



»BH¥ 1 1 -8 8 9 1 3 

14 

7^S 1 4T<DWWfT. /^7^WfiSPo^yi/-t 
<ffl (#2yi/-tl) K&^^tCti, Xf7^S3 0 
aT**fT"T«». Xf7^S3 0-Xf7yS4 0H 
Xf7ys 1 6-Xf7^S 2 6 t**«^ttfP»4RIi; 

[0068] *>< it, y-A^stfrWffittsaan 

-5. y-A<7>jgfT«J8t4, /W^£>ftB, Xf>f7^<0 
fiIT$0, WfJCUfci-Jt, H«£fiESS5 0 5 0 (5 
0 5 0L, 5050 R) KAlfcSnSo 

m&&m<DffiiE) mi 6m\t, mmmm^ 5040 

(5 0 4 0 L, 5040 R) \Z&VtZ>i&m<?>Mffl¥M<n 

(4BS.r/®S$^»^-^^, CCD*^72 4 0^6# 
6n&iS«^<0^— #&BK:<k?K *^72 4 00)fif 

:-(-as ta b \z&m \zm % * L S;*:9?j?* r *-^)"^W3E* r 

< >#&&fTm&&^%Z-£\z&K)ffimir&'b<D'V$> 

[0 0 6 9] IP'S, Xf7^S4 0 0m 
2 2 0Otii*l:S^t, */5a>bT3. — f>yaEtfefr 
?>J (4X4) ^rfh^-r^o Xf7^S4 10t!H # * 

^ 2 4 0 ©a«wa«**ff j< (8E«» &^&v-^<d 

^JW-TS. V — *&B«W855 0 6 0 (5 0 6 0 L, 5 

0 6 or) \t. y I/ - 1 ©R»l:» o #w snt* ^ 7 

2 4 0 (2 4 0 L, 2 4 0 R) S^fcHflfe^TV-"-* 
*i6*LT^£. S£oT, V-;*jffiB&tBgB5 0 6 0*4 

^mu/cv-*feB^, miEmmffi 5 0 4 0 uf*^ 

S 4 2 0C*^T) ^JK-To fflxE®agg5 0 4 0 (5 0 

4 0L, 5 0 4 0 R) tt, Xf7yS4 2 0l:^5^T, 

5 4 3 0 Tt4, */7 2 4 0 <D {£ B O *fi IE <B £ ft U 
T» Z:<DffiTEmtzm~J^T. Xf7yS440T, 

<fc£fifcgi$5 0 5 0 (5 0 5 0L, 5050 R) tzSTTo 
[0 0 7 0] * 1 7Httv-*ffiK«ttiffl5 0 6 0 tC*5 
WSffla^JaTftS. Xf77 p S 5 0 0Tii #;<^2 

4 d*^»L,/t*^—Bi«*Bl0jitro Xf 
7^S 5 0 2tll rp — Sr, Xf7^ 

5 5 0 6T« f^D — A;HB««5Rj £rfToT, 
J««*l;:,fcoT*3nfcv — #ttfi (x. y) Sr^UJT 
£> e Xf77 r S 5 0 2(0 rn- #;MH«1S55Kj . Xf7 
yS 5 0 6(D r^D-A;P«««*J tt, ¥HK£UT 
tt, vu^ h«tRjc*5tts rp— *^««a»j (SB 9 
El) . ryawWWMBJRj (S10H) i:*K»k: 

[0 0 7 1] T-*«*(Dfc»<Z)»*« Is (X 

f7^S 2 3 2<?)) tit, (&) lC^>t> 
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[0 0 7 2] 
Kill 



[0 0 7 6] 
[Rl 3] 



[0 0 7 3] Sffl^^.-yu— V# 1 fctP^Tfi, V- 
# (1600-1604) l:tt*fiS:ffl^T^5<0T, 

2 C6) KO^TttSfiO)^ — # (1 650-165 
4) *ffl^T^£<B7?, 

[0 0 7 4] 

[Rl 2] 

G 

(R + B) / 



~hx'~ 




X 


by' 


= M 


y 


h 




i 



[00 '75]^^^ Sfc, jfo— /WffljRfcl**** 
4#a&* I s (x, y) to^T6±E2^0lS«^ 
Xf7^S 5 0 2R^ry^S 5 0 6-C#<E>n£: 

"cosr -sinr 0 



[0 0 7 7] T££o ^^(C, ?g 1 6 0CD7^;/:/S 4 
1 0 <D!&m<Dftmz-D^T* 8l8HSffl^T8!Wt 

*>S#*5ffi8t*^<BSE*fTJ8Mc (4X4£0ha — f 

SH»ffil8*^<0£»fT?UPc (4X4) fcgEfcI(E>«I<h 

^es (x, y, z) t)itftitiT^enTv^o 

- --[ 0 0 7" 8 ]-J^»;^"3"^; "ftft r £ #7-^ 2"4~0 <0{4 
B7?OZ$fi;£fa<£{Hj&(roll)<hU Mp^^724 
0<7>&gTCQX$fi:£ft<9Ee(pitch)<hU 
^724 OtDffiSTCOZtt^fplOlHlKCyaw)^^^^, 

2? [0 0 7 9] 
[Rl 4] 



Mr = 



sin r 
0 
0 



cosr 0 
0 1 
0 0 



0 
cosp 
sinp 

0 



0 

—sinp 
cosp 
0 



" cos$ 


o 


sia<|> 


0" 


"1 


0 


0 


-X" 


0 


1 


0 


0 


0 


1 


0 


-Y 


-sin<> 


0 


COS<|> 


0 


0 


0 


1 


-z 


0 


0 


0 


1 


0 


0 


0 


1 



[0 0 8 0] d£#*9 2 4 OCD^EBL w 

[0 0 8 1] 
[Rl 5] 

"a/w ooo" 

0 d/h 0 0 
Pc " 0 0-10 
0 0-10. 

[0 0 8 2] 7?*Sn*. Xr^^S 5 2 OTit tt@ 
V-^OffiJU&It (X, Y. Z) ft, ^SCtCttoT. S 
&ffi«^7?<Dfiit (Xh. yi, z„) 

[0 0 8 3] 

[Rl 6] 







X" 




= P C M C 


Y 
Z 


i 




1 



[0 0 8 4] Xfy^S 5 2 2T(1 
40 figx, y <h IT, 
[0 0 8 5] 
[Rl 7] 




[0 0 8 6] *f»*. Xfy^S4 2 0l:*tt 
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0 0 0±K45HT* -^^1/^*^7 2 4 
0*«fi[6 0 0^Mt/:i^^1"o f-^HOO 
0±Ki£#£*XTV>£>V--#2\ M^.tfMi-M7<hU 

-So H*fe6 0 0rttC«> T-^M 2 , Ms, M«, M 7 ^t 

5 2 0tf*»6fttfe©T»0, ^tl^Pi <h-r 
3. *fc, Qtt* 7-*ttl*IB»5 0 6 0««*lHU 

Stages o 6 oa*s«*ftfc-7— a&fiSrs-r. 

[0 0 8 7] 7fyyS 4 2 0© rv— *0«gijj 
V-#&itJ&ffig&5 0 6 0«tillfcV-*filQ^ 

T<fc£o % 1 9Hi:*^T, ^ h;u ei ^tti^n 
— S-4-2-0-©ft|«*^^ 

" Itt, Ii6 0 0 0l*Ii;A§V^i (i=0 — n) <D 

[0 0 8 8] 
»1 8] 

i :Min{e { ] 

[0 0 8 9] »1.9B^rat PaMTOlE 

[0 0 9 0] (ft, Xr^^S4 3 0, Xf7^S44 0 

. - - - 

-r-SfflS^ fgi 7 0i:gfJ;5i:, *«3fc:jt,oa>oJfc 

Sfl2 1 EHCST cfcifc* *§7l/-A*CV-^M. 
<0flttfi«8 1 OlCtt, v-*7&<««8 1 0<OSH«"Ctt 

8 i ort^-&*nT^*«-&fctt, 

[0 0 9 11 COiifcKIIElCttJH-rST-AeWO* 
AftCfcOUBjiltLT, *OW0»*> ttlE 
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[0 0 9 2] IP*>, *S7l^-AT<Z)1iiEffl[ 

v* '*«ffc3te*tjE«[4:-r*. 
[0 0 9 3] 
i0 [» 1 9 ] 

v' 1 = av' t_I + (l-a)-v l 
[0 0 9 4] ::ta(l »*<B«ffl<0#»<offi£-^£ 
^st-r-5 0^a<l(D5£3gcT*3o ±Kit<Z>j|cWrr*£: 

il, ^[HlcD7 U — ATSSnfcMiEffl: v» £ (1- 
[0 0 9 5] (X»fl2) ±«B*«S»IB-Ctt, 

30 Tfe. V-*^fttj^U-A^^i0Eb/ u c{4g^^^:^<~r 
[ 0 0 9-6 ] fg 2-2 H-a, -mm<rj7*\/—tt\Z*5^XJL~3 

o fc v - * ^^ecd 7 a £ Ms ^rift^-r 

[0 0 9 7] <£»«4> ±E*Dfc«ffitt* XT*y^r 
*tf^U!yh»fp) l^<Dy?^^^©{Cci:0}t^b 

50 [0 0 9 8] *5&W<D. *ftfl!)V-*K:tt^fc«S8* 



19 

[0 0 9 9] 
[BB<OfB*ttBW!] 

[HI] fl£*a«fc*^T, #<fctf*5i^«D35MWMB 

Ho 

[B 2] *58W(0*JK»JBlcffl^6nT^*y"-AK 

[B 3 ] US 2 H^y- Aft«T?aE«^U— ^©a*»c 

.^A.S_>=L^LS:.«ttBB-il*.H.- - — 

[0 4] m2m<D?-Amwizm\*t>nT^z>HMD 
[05] ^2i^coy-A^g<D^-^Kcie^^n^ 

[H6] S5H©f-^;H^ " ^O^KjtCO 
[H 7 ] ^IfiJBffioy— ASBKOfcAO, H#c5tHH& 

[B8] mffimmwuv httwi-»8i:Bs»if 
[B9] Htt««-aia5tHaaa# 

[HI] 




11) 4*§B¥ 1 1 -8 8 9 1 3 

[bio] mmmm<D^^y biSLWWrmmzmzi&m 

[HI 1] S8i^)7D-ft- h<Z>fflHl::*3^TJ8 

[HI 2] ^SBco^n-^^- hCD^Stc^^Tffl 
i> e>n*»*«*<z>S8£^i£&»?"B. 
i0 [0 13] **JE»Sg©V-Afw43ttSfiffiy-A7 

- ;v k cdjKj5E £ m m -r £ b . 
[B i 4 ] *jn»i8©y- 

A^^^J^JS^^T^^p-^-V*- ho 
[HI 5] httfflfcfe^f SSIftKtSB. 

[@i6] iiMjgiifcistj-sttiEfflaBBoffla^ies: 

^WjI-ttt^^P:-^ 1- -:hv — — — — — ~ 

[HI 7] "«r6BO7P-ft-K0-9 (V— * 
0>iitt) ^f^iffltClftBJ-r^^P-^^-ho 
[HI 8] mi 6i(Z)7D-ft- hO— SB 

ttioTa) ^i^jffl^iftHj-r^^p-^^-ho 

[Hi 9] *IEa>:fcAKttfliStt*i£*fcfc*^~rt 

cD^mco^^^-r^Bo 

[H2 0] s«pt&-5-7-*©Ji*ai<z>JSa*Kw-rs 

^7 P — — ho 
[B2 1] SI^IBORJBW^afflSftS**^-* 

[B2 2] mmmm<D^mm^zmm^n^-timm 

OBCll*KWr*B. 

[04] 




(12) 



nm¥ ii-889i 



[0 2] 



1 5] 



2000 



220L 



CGHfc 



240L / 




y v 



250 



-230 



220R 3000 



V 21 OR 





width 



[03] 



[05] 



21 OR 



1200R 




1650 1651 1652 

n □ □ 



1603 1604 

o o 



1653 1654 

□ □ 



1600 1601 1602 

o o o 



[0 6] 



1000 




IL 



1 1] 



IR 



[02 0] 

for each 1 (i =0—n) 



1000 













PiQl 


m NO 


a < mill ? 






[yes 




min *= o 
Marker «l 






1 



(13) 



#M¥l 1-88913 




(15) 



ftfflVl 1-88913 



[0 91 
©ft I 



x Cl'x - A, I'x ♦ A) 
y cry - B. IV + B) 



S222 



S230 

4- 




2B + 1 




f S234 



N = N 


\ 1 


Sumx 


= Sumx i x 


Sumy 


= Sumx * y 



1 8] 



S236 




NO 



YES S238 



Ix = Sumx/N 
Iy — Sumy/N 



(Ix, Iy) £ return 



return 

"Not Found" 



IMcPc 



S520 



yh 


= P C M C 


"X" 

Y 
Z 


Zh 
. 1 - 




1 



return (x, y) 



[^2 2] 



(16) 



4$BH¥ 1 1 - 8 8 9 1 





Return 
"Not Found' 



YES 



1 fcCDyis AGO 

Return (he, Iy) - p _ /u^iiffiffi 




« 



(17) 



1 1-889 



1 4] 



(" START ) 



S10 



SI 4 




/<?^ {Mid* 



NO 




S30 



S20 



NO 



S24 



NO 



NO 



-S18 



"YES 



± 




S32 



L 




S34 



yes S22 S36 yes 





yes S26 S40 





1 



S50 



S52 



(18) 



<&m¥- 1 1 -8 8 9 1 



1 6] 



1 



S400 



i 



S410 



S420- 



.__S430, 



S440; 



220 



* y v <E>a&ttS&£8tfT$i| 



J&&^ 











T 



5060 



[021] 




(19) 



11-88913 



[01 7] 

S500 



— S502 



S504 




YES WU£M& t 
Cx, y) 



1 


NO 

r {S506 






r S508 




YES Bfjfe^g 

(x. y) 




return 

"Not Found" 



(72) UJ# *S£ 

^TfTI5K^Br6Tgl45Slfe 



(72)&9i^ ®* S*a 

«fifljBIZ*e$OT6TB145#Jtt «SS*7£ 



